Proximal methods for sparse optimal scoring and discriminant analysis

Linear discriminant analysis (LDA) is a classical method for dimensionality reduction, where vectors are sought to project data lower dimensional space optimal separability of classes. Several recent papers have outlined strategies, based on exploiting sparsity the vectors, performing LDA in high-dimensional setting number features exceeds observations data. However, many these proposed methods lack scalable solution underlying optimization problems. We consider an scheme solving sparse scoring formulation block coordinate descent. Each iteration this algorithm requires update vector, which admits analytic formula, and corresponding convex subproblem; we will propose several variants proximal gradient or alternating direction multipliers used solve subproblem. show that per-iteration cost scales linearly dimension provided restricted regularization terms employed, cubically worst case. Furthermore, establish when descent framework generates convergent subsequences iterates, then converge stationary points problem. demonstrate effectiveness our new with empirical results classification Gaussian sets drawn from benchmarking repositories, including time-series multispectral X-ray data, provide Matlab R implementations schemes.